System and method for validating geospatial data collection with mediated reality

ABSTRACT

There is provided a system and method of validating geospatial object data with mediated reality. The method including: receiving an object definition associated with a geospatial object, the object definition comprising a type of object and a position; displaying a visual representation of the geospatial object to a user relative to a corresponding geospatial object located in a physical scene; and receiving input validating a placement of the visual representation relative to the corresponding geospatial object located in the physical scene.

TECHNICAL FIELD

The following relates generally to geospatial data management; and moreparticularly, to systems and methods for validating geospatial datacollection with mediated reality.

BACKGROUND

Surveying firms, mapping firms, municipalities, public utilities, andmany other entities, collect, store, use, and disseminate vast amountsof geospatial data. This geospatial data can be used to manage dailyoperations and conduct mission-critical tasks; for example, assetmaintenance, construction plan design, zoning proposals, among manyothers. Traditionally, geospatial data is collected using manualmeasurements (offsets) from detectable local landscape features; forexample, a curb line. Then the collected measurements would be plottedon a map to indicate object/asset locations. The maps could then bereprinted for use in the field. While much of this geospatial data canbe digitized, the accuracy and quality of such digital representationsmay affect the tasks and applications that rely on such data. In otherapproaches, location tools, such as global navigation satellite systems(GNSS) and/or real-time kinematic (RTK), can be used to collect digitalgeospatial data. These approaches generally require cumbersome,unsophisticated, and time-consuming validation techniques.

SUMMARY

In an aspect, there is provided a computer-implemented method ofvalidating geospatial object data collection with mediated reality, themethod comprising: receiving an object definition associated with ageospatial object, the object definition comprising a type of object anda position; displaying a visual representation of the geospatial objectto a user relative to a corresponding geospatial object located in aphysical scene; and receiving input validating a placement of the visualrepresentation relative to the corresponding geospatial object locatedin the physical scene.

In a particular case of the method, the position of the geospatialobject in the object definition comprises latitude and longitude.

In another case of the method, the position of the geospatial object inthe object definition comprises a point associated with the geospatialobject.

In yet another case of the method, the position of the geospatial objectis determined using at least one of global navigation satellite systems(GNSS) and real-time kinematic (RTK) positioning.

In yet another case of the method, receiving input validating theplacement of the visual representation comprises receiving aconfirmatory input from a user.

In yet another case of the method, receiving input validating theplacement of the visual representation comprises receiving aconfirmatory output from machine vision and artificial intelligencetechniques.

In yet another case of the method, the method further comprisingrecording, in the object definition, the validated position of thegeospatial object.

In yet another case of the method, the object definition furthercomprises one or more attributes associated with the geospatial object,and the method further comprises receiving input validating one or moreof the attributes associated with the geospatial object.

In yet another case of the method, the method further comprisingrecording, in the object definition, the one or more validatedattributes associated with the geospatial object.

In yet another case of the method, the method further comprisingassociating an image of the physical scene with the validated position.

In another aspect, there is provided a system of validating geospatialobject data collection with mediated reality, the system comprising oneor more processors and data storage memory in communication with the oneor more processors, the one or more processors configured to execute: anobject module to receive an object definition associated with ageospatial object, the object definition comprising a type of object anda position; a display module to display a visual representation of thegeospatial object to a user relative to a corresponding geospatialobject located in a physical scene; a validation module to receive inputvalidating a placement of the visual representation relative to thecorresponding geospatial object located in the physical scene.

In a particular case of the system, the position of the geospatialobject in the object definition comprises latitude and longitude.

In another case of the system, the position of the geospatial object inthe object definition comprises a point associated with the geospatialobject.

In yet another case of the system, the position of the geospatial objectis determined using at least one of global navigation satellite systems(GNSS) and real-time kinematic (RTK) positioning.

In yet another case of the system, receiving the input validating theplacement of the visual representation comprises receiving aconfirmatory input from a user via an input device.

In yet another case of the system, receiving input validating theplacement of the visual representation comprises receiving aconfirmatory output from machine vision and artificial intelligencetechniques.

In yet another case of the system, the system further comprising arecordation module to record, in the object definition, the validatedposition of the geospatial object.

In yet another case of the system, the object definition furthercomprises one or more attributes associated with the geospatial object,and wherein the validation module further receives input validating oneor more of the attributes associated with the geospatial object.

In yet another case of the system, the system further comprising arecordation module to record, in the object definition, the one or morevalidated attributes associated with the geospatial object.

In yet another case of the system, the system further comprising arecordation module to record an image of the physical scene associatedwith the validated position.

These and other aspects are contemplated and described herein. It willbe appreciated that the foregoing summary sets out representativeaspects of the system and method to assist skilled readers inunderstanding the following detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

A greater understanding of the embodiments will be had with reference tothe figures, in which:

FIG. 1 illustrates a block diagram of a system of collecting geospatialobject data with mediated reality, according to an embodiment;

FIG. 2 illustrates a flow diagram of a method of collecting geospatialobject data with mediated reality, according to an embodiment;

FIG. 3A illustrates an exemplary image of collecting geospatial data byplacing an antenna;

FIG. 3B illustrates an exemplary diagram of collecting geospatial databy placing an antenna;

FIG. 4 illustrates exemplary screenshots of two-dimensional maps showinggeospatial objects;

FIG. 5 illustrates an exemplary image of a user validating geospatialobjects using the system of FIG. 1;

FIG. 6 illustrates an example screenshot of a visual representation of areceived geospatial object over a captured scene, in accordance with thesystem of FIG. 1;

FIG. 7 illustrates an example screenshot of validating placement of thevisual representation of the object of FIG. 6, in accordance with thesystem of FIG. 1;

FIG. 8 illustrates an example screenshot of recordation of the verifiedplacement of the object of FIG. 6, in accordance with the system of FIG.1.

DETAILED DESCRIPTION

Embodiments will now be described with reference to the figures. Forsimplicity and clarity of illustration, where considered appropriate,reference numerals may be repeated among the Figures to indicatecorresponding or analogous elements. In addition, numerous specificdetails are set forth in order to provide a thorough understanding ofthe embodiments described herein. However, it will be understood bythose of ordinary skill in the art that the embodiments described hereinmay be practiced without these specific details. In other instances,well-known methods, procedures, and components have not been describedin detail so as not to obscure the embodiments described herein. Also,the description is not to be considered as limiting the scope of theembodiments described herein.

Various terms used throughout the present description may be read andunderstood as follows, unless the context indicates otherwise: “or” asused throughout is inclusive, as though written “and/or”; singulararticles and pronouns as used throughout include their plural forms, andvice versa; similarly, gendered pronouns include their counterpartpronouns so that pronouns should not be understood as limiting anythingdescribed herein to use, implementation, performance, etc. by a singlegender; “exemplary” should be understood as “illustrative” or“exemplifying” and not necessarily as “preferred” over otherembodiments. Further definitions for terms may be set out herein; thesemay apply to prior and subsequent instances of those terms, as will beunderstood from a reading of the present description.

Any module, unit, component, server, computer, terminal, engine, ordevice exemplified herein that executes instructions may include orotherwise have access to computer-readable media such as storage media,computer storage media, or data storage devices (removable and/ornon-removable) such as, for example, magnetic disks, optical disks, ortape. Computer storage media may include volatile and non-volatile,removable and non-removable media implemented in any method ortechnology for storage of information, such as computer-readableinstructions, data structures, program modules, or other data. Examplesof computer storage media include RAM, ROM, EEPROM, flash memory orother memory technology, CD-ROM, digital versatile disks (DVD) or otheroptical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which canbe used to store the desired information, and which can be accessed byan application, module, or both. Any such computer storage media may bepart of the device or accessible or connectable thereto. Further, unlessthe context clearly indicates otherwise, any processor or controller setout herein may be implemented as a singular processor or as a pluralityof processors. The plurality of processors may be arrayed ordistributed, and any processing function referred to herein may becarried out by one or by a plurality of processors, even though a singleprocessor may be exemplified. Any method, application, or module hereindescribed may be implemented using computer readable/executableinstructions that may be stored or otherwise held by suchcomputer-readable media and executed by the one or more processors.

The following relates generally to geospatial data management; and moreparticularly, to systems and methods ‘for validating geospatial datacollection with mediated reality.

While the following disclosure refers to mediated reality, it iscontemplated that this includes any suitable mixture of virtual aspectsand real aspects; for example, augmented reality (AR), mixed reality,modulated reality, holograms, and the like. The mediated realitytechniques described herein can utilize any suitable hardware; forexample, smartphones, tablets, mixed reality devices (for example,Microsoft™ HoloLens™), true holographic systems, purpose-built hardware,and the like.

Advantageously, the present embodiments employ advanced visualizationtechnologies, such as mediated reality techniques, to work inconjunction with other data collection techniques to provide immediatevisual validation of data collection accuracy.

In some cases, survey-grade data collection can be accomplished withgeographic information systems (GIS) using high-precision globalnavigation satellite systems (GNSS) and/or real-time kinematic (RTK)positioning, to capture location of assets or points for geospatial datacollection. As illustrated in the example of FIGS. 3A and 3B, thecollection of geospatial data can be accomplished by placing a GNSSantenna near or on top of a placemark and then recording, for example,latitude, longitude and elevation of the antenna; thus, the ‘x, y, z’geospatial coordinates of the placemark in two-dimensional (2D) orthree-dimensional (3D) space. In some cases, complimentary tools, forexample laser mapping, can further enhance collection by allowingcollection of data of hard-to-reach objects.

As illustrated in the example screenshots of FIG. 4, in some approaches,captured geospatial data can be displayed on a 2D map to help asurveyor, or other user, validate accuracy of the data. In some cases,the captured information can be transferred to a server or storedlocally for review. In some cases, the elevation data and otherattributes of the object can be stored as part of the capturedgeospatial data metadata in text format.

In many cases, in order to ensure accuracy of the data, the surveyor isrequired to check the collected data points and objects multiple times.This requirement can result in substantial inefficiencies for the user,and potentially add substantial time and cost to projects; especiallywhere many objects are to be collected.

Advantageously, embodiments of the present disclosure can substantiallyimprove efficiencies in accuracy verification of collected geospatialobjects. Immediately after a data point on a geospatial object iscollected, embodiments of the present disclosure can use mediatedreality techniques to display a virtually representation of thecollected object relative to its physical space. Thus, enabling the userto easily and immediately validate the accuracy of the collected object.

Advantageously, embodiments of the present disclosure can providereal-time visual validation of geospatial object placement in, forexample, three-dimensional (3D) space (e.g., latitude, longitude, andelevation). Such embodiments can improve the quality and accuracy ofcollected geospatial objects, and increase the speed of geospatialobject data collection. In some cases, the present embodiments can beused to supplement other tools and approaches for geospatial datacollection. In doing so, the present embodiments can reduce the cost ofdata collection by reducing the time needed for data capture and qualitycontrol.

Embodiments of the present disclosure can provide verification bygenerating a geospatial image overlaid over in an image or video of ascene captured by a camera (for example, as in augmented reality) ordisplayed as a hologram (for example, as in mixed reality or holographicsystems). This can be performed in a manner that anchors to realitythrough geographical positioning, thereby generating a geographicallyrelevant composite image or a hologram that can be presented to a user.

Embodiments of the present disclosure can advantageously provide athree-dimensional model or a raster symbol of real time data viewable,for example, on mobile devices, wearable devices, or other viewingplatforms. The geospatial images can be used to provide real-time visualrepresentations to perform geospatial data verification of datacollected via other approaches.

Turning to FIG. 1, a system of validating geospatial data with mediatedreality 150 is shown, according to an embodiment. In this embodiment,the system 150 is run on a local computing device (for example, a mobiledevice). In further embodiments, the system 150 can be run on any othercomputing device; for example, a server, a dedicated price of hardware,a laptop computer, a smartphone, a tablet, mixed reality devices such asMicrosoft™ HoloLens™, true holographic systems, purpose-built hardware,or the like. In some embodiments, the components of the system 150 arestored by and executed on a single computing device. In otherembodiments, the components of the system 150 are distributed among twoor more computer systems that may be locally or remotely distributed;for example, using cloud-computing resources.

FIG. 1 shows various physical and logical components of an embodiment ofthe system 150. As shown, the system 150 has a number of physical andlogical components, including a central processing unit (“CPU”) 152(comprising one or more processors), random access memory (“RAM”) 154, auser interface 156, a device interface 158, a network interface 160,non-volatile storage 162, and a local bus 164 enabling CPU 152 tocommunicate with the other components. CPU 152 executes an operatingsystem, and various modules, as described below in greater detail. RAM154 provides relatively responsive volatile storage to CPU 152. The userinterface 156 enables an administrator or user to provide input via aninput device, for example a mouse or a touchscreen. The user interface156 also outputs information to output devices; for example, a mediatedreality device 192, a display or multiple displays, a holographicvisualization unit, and the like. The mediated reality device 192 caninclude any device suitable for displaying augmented or mixed realityvisuals; for example, smartphones, tablets, holographic goggles,purpose-built hardware, or other devices. The mediated reality device192 may include other output sources, such as speakers. In some cases,the system 150 can be collocated or part of the mediated reality device192. In some cases, the user interface 156 can have the input device andthe output device be the same device (for example, via a touchscreen).

The network interface 160 and/or the device interface 158 permitscommunication with other systems or devices, such as other computingdevices and servers remotely located from the system 150. The deviceinterface 158 can communicate with one or more other computing devices190 that are either internal or external to the system 150; for example,a GNSS device to capture a position and/or elevation, a camera or cameraarray to capture image(s) of a scene, sensors for determining positionand/or orientation (for example, time-of-flight sensors, compass, depthsensors, spatial sensors, inertial measurement unit (IMU), lasermapping, and the like). In some cases, at least some of the computingdevices 190 can be collocated or part of the mediated reality device192. In some embodiments, the device interface 158 can receive and senddata to other devices, such as positions, elevations, and images, whichhave been previously captured, from the local database 166 or a remotedatabase via the network interface 160.

Non-volatile storage 162 stores the operating system and programs,including computer-executable instructions for implementing theoperating system and modules, as well as any data used by theseservices. Additional stored data can be stored in a database 166. Duringoperation of the system 150, the operating system, the modules, and therelated data may be retrieved from the non-volatile storage 162 andplaced in RAM 154 to facilitate execution.

In an embodiment, the system 150 further includes a number of modules tobe executed on the one or more processors 152, including an objectmodule 170, a position module 172, a display module 174, a validationmodule 176, and a recordation module 184.

Turning to FIG. 2, a flowchart for a method of validating geospatialdata with mediated reality 200 is shown, according to an embodiment.

At block 202, the object module 170 receives an object definitionassociated with a geospatial object that has been collected by acomputing device 190 comprising a geographic information system (GIS).The geospatial object data corresponds to a geospatial object physicallylocated in space and the collection of the geospatial object data can beaccomplished using any suitable approach. In an example, a usercollecting geospatial object data can place a GNSS and/or RTK on top ornear a point associated with the object they want to collect. In othercases, the user can use a laser mapping device to remotely determine adistance and elevation to such point; in some cases, using GNSS and/orRTK for positioning information as an anchor. After identification ofthe position and/or elevation of the object, this information can bestored as part of an object definition for the geospatial object. Theobject definition can be stored locally on the respective computingdevice 190, stored on the database 166, or stored remotely (for example,a cloud-based or server-based repository) and communicated via thenetwork interface 160.

The object definition includes the type of object (for example, a pipeor a point) and the geographical coordinates of its physical position.In further cases, the object definition can also include attributes orcharacteristics of the object. In most cases, the geographicalcoordinates are relative to the surface of the earth, for examplelatitude and longitude. In other cases, the geographical coordinates canbe relative to another object; for example, relative to a building orlandmark. In some cases, the object definition includes otherproperties; for example, an elevation, an object type, an object size,an object orientation, a material type, and the like.

In some cases, the object definition can include representationinformation for the geospatial object. In some cases, the representationinformation can include a point, line, or area associated with thephysical position of the geospatial object. In other cases, therepresentation information can include information required for creatingmore sophisticated 3D visuals; for example, geometry type, a 3D model,an object type (such as hydrant or manhole), object condition, colour,shape, and other parameters. In some cases, the representationinformation can be determined by the computing device 190 comprising thespatial sensor and included in the object definition sent to the system150. In other cases, the representation information can be determined bythe object module 170 upon receiving the object definition. For example,by referencing the properties of the object in the object definition. Asan example, the object definition can include: a manhole 1.2 m wide and3.2 m deep with grey cover installed in 1987 oriented 14d North.

In some cases, the object module 170 can receive the object definitionusing “push” or “pull” approaches; such as over an applicationprogramming interface (API). In some cases, the format of the objectdefinition can include GeoJSON or other protocols.

In some cases, once the user captures a position of a geospatial object,the object definition is automatically sent to the object module 170 andmethod 200 proceeds. In other cases, once the user captures a positionof a geospatial object, the user is given the option to proceed with themethod 200 and thus send the object definition to the object module 170.

At block 204, the position module 172 receives or determines a physicalposition of the system 150 from a computing device 190 comprising aspatial sensor; where the physical position includes geographicalcoordinates. In most cases, the geographical coordinates are relative tothe surface of the earth, for example latitude and longitude. In othercases, the geographical coordinates can be relative to another object;for example, relative to a building or landmark. In some cases, thephysical position includes an elevation. In some cases, the positionmodule 172 also receives or determines an orientation or bearing of thesystem 100; for example, comprising the physical orientation of thedirection of the camera. In an example, the position module 172 candetermine the position and orientation in 2D or 3D space (latitude,longitude, and, in some cases, elevation) using internal or externalspatial sensors and positioning frameworks; for example, global positionsystem (GPS), GNSS and/or RTK, Wi-Fi positioning system (WPS), manualcalibration, vGIS calibration, markers, and/or other approaches. Theposition module 172 can then track the position and/or the orientationduring operation of the system 150. The physical position is used by thesystem 150 to, as described herein, accurately place the visualrepresentation of the geospatial object displayed to the user relativeto the physical space using the position of the geospatial object in thephysical space in the object definition.

At block 206, in some cases, the display module 174 displays a mediatedreality ‘live’ view (such as a video stream or a sequential stream ofcaptured images) received from a camera. This live view is oriented inthe direction of the system 150 as received by the position module 172in block 202. In embodiments using holographic devices, in some cases,receiving the ‘live view’ can be omitted because the visualrepresentation itself is displayed in the physical space.

At block 208, the display module 174 presents a visual representation tothe user via the user interface 156, where the visual representation isa representation of the received object definition. The objectdefinition includes has spatial attributes (for example, latitude,longitude, and elevation), which the display module 174 can use, inconjunction with the physical position information from the positionmodule 172, to place the visual representation relative to the capturedscene. For example, placing the visual representation of the objectoverlaid onto the live view. In most cases, the object definition can beused to scale the visual representation according to the 3D perspectiveof the live view.

The visual representation can be, for example, a three-dimensional (3D)digital-twin model resembling the collected object. In further cases,the visual representation can be, for example, a symbol representing theobject, such as a point, a flag, a tag, or the like. In further cases,the visual representation can be, for example, a schematicrepresentation, a raster image, or the like. In some cases, the type ofvisual representation can be associated with the object in the library;and in other cases, the type of visual representation can be selected bythe user.

In some cases, for example where the visual representation is anythingother than a 3D model of the object (for example, a manhole symbolizedby a point or a flag), a key location (for example, the point or thebase of the symbol) can be placed at a respective key point of thephysical object captured by the camera (for example, at the center ofthe manhole). In some cases, the symbology for each object, as well asthe key locations and points, can be defined by each user.

In some cases, along with the visual representation, other informationcan be displayed; for example, distance, elevation, size, shape,colours, and the like, can be displayed to assist with visualizationand/or precise placement. In some cases, such as with GIS, the visualrepresentation location can be represented by a single point, line, oroutline, and to help the user understand where the object is placed, apoint, a cross, a line, or other means, can be used within the visualrepresentation.

In other cases, the display module 174 can stream the visualrepresentation (for example, a 3D model or model rendering) from aserver, cloud-based infrastructure, or other external processing device.Instructions for such streaming can be provided in any suitable format(for example, KML or GeoJSON) or any other proprietary format.

FIG. 5 illustrates an example of a user (surveyor) validating theposition of piping of a fire hydrant. In this example, the system 150 islocated on a tablet computer. The object module 172 has receivedgeospatial object data for the piping of the fire hydrant and thedisplay module 174 displays a 3D model as a visual representation of theobject on the display of the tablet. The visual representation isdisplayed by the display module 174 overtop of a live view that is beingcaptured by the rear-facing camera of the tablet.

In some cases, the position of the object represented by the visualrepresentation by the display module 174 can be accurately determined;for example, by the position module 172, by using the position of thesystem 150 (for example, its latitude, longitude, and elevation), anazimuth of the system 150, and the distance to one or more objectscaptured by the camera. In some cases, the position module 172 cancapture metadata from the GNSS and/or RTK device, and then correct itfor the elevation and distance difference between the GNSS antenna andthe presented visual representation to achieve survey-grade dataaccuracy. In some cases, the user can update the position and/or theproperties of the visual representation manually.

In an example, when the position module 172 determines the system's 150location and orientation in x,y,z space, the position module 172 alsodetermines the position and orientation of the physical camera (x,y,zplus bearing). The display module 174 can use the positioninginformation to access spatial data (the data with x,y,z coordinates) andcreate (or use an existing) visual representation (for example, a 3Dmodel). The display module 174 can place the virtual camera in thelocation of the physical camera (x,y,z plus bearing) relative to thevisual representation. The visual representation can be overlaid on topof the physical representation such that it can appear in the correctlocation, matching physical objects around it. In this way, byunderstanding x,y,z and orientation of the physical camera, the displaymodule 174 can display visual representations of objects that are in thescene (or field of view), and size and orient the visual representationto allow for visualization that matches the physical world accurately.

In some cases, distance to the physical geospatial object can bemeasured using distance finders. The distance finder may also detect theobject's elevation (either relative or absolute). Examples of otherdistance determination approaches can include optical tools, such asdepth cameras or time-of-flight sensor, or image processing thatcompares images taken from multiple locations or angles to determinedistances. These other approaches can be used separately, or they can beconnected or associated with the system 150 to provide informationautomatically. For example, the display module 174 may displaycross-hairs and, upon aligning the cross-hairs with the physical object,the system 150 can send a request to a distance finder to determine thedistance to that point. In another example, the user interface 156 canreceive an indication from the user of a point they want to measure thedistance to. In another example, an external distance finder can be usedto determine the distance to the physical object separately, and thatdistance can be used to ensure accurate object placement by displayingthe distance to the collected object.

At block 210, the validation module 176 receives input with respect tovalidating the position and attributes of the visual representationrelative to the corresponding geospatial object captured in the scene.If visual validation shows discrepancy in alignments and/or any of theassociated attributes of the geospatial object and the correspondingvisual representation, the user, via the user interface 156, canindicate that there is a discrepancy, and/or indicate that there is adiscrepancy with any of the associated attributes.

In some cases, the input received by the validation module 176 couldinclude outputs from machine vision (MV) and/or artificial intelligence(AI) techniques. Such techniques can be used by the display module 174to automate display of the visual representation. In an example, MV andAI techniques can automatically detect a geospatial object in thecaptured scene, then validate whether the placement of the visualrepresentation is aligned with the geospatial object in the capturedscene. In an example, the outputs from the MV and/or AI techniques candetermine the correct position of the visual representation by snappingthe visual representation to the corresponding geospatial objectcaptured by the camera (for example, in the example of FIG. 7, snappingthe diameter of the pipe to the diameter of the manhole). If thisposition is different than the position indicated in the objectdefinition, the validation module 176 can indicate a discrepancy,otherwise it can validate the position. In some cases, the machinevision technique can be used to auto-size the visual representation tothe geospatial object captured by the camera; then, the user canvalidate the position.

In some cases, the input received by the validation module 176 couldinclude corrections to attributes or properties (as described herein)associated with the geospatial object; either inputted by the user orautomatically detected by the MV and/or AI techniques. For example,corrections to the object type, object size, colours, shape, elevation,rotation, object conditions (e.g., rust or damages), and the like.

In some cases, =the visual representation can be temporarily fixed inplace so that the user can look at it from different angles to confirmthe location.

At block 212, the recordation module 178 records the validated positionof the respective geospatial object as the position of the correspondingvisual representation in the captured scene. In some cases, therecordation module 178 also records validated attributes and/orproperties of the geospatial object in the object definition (forexample as metadata); for example, size, height, orientation, GNSSand/or RTK information, and the like. The recordation can includeediting the object definition in a geospatial data storage; for example,on the database 166 or sent to an external storage via the networkinterface 160. In other cases, a new revised entry can be stored in thegeospatial data storage. In some cases, the recordation module 178 canrecord an image of the physical scene in association with the validatedposition; where this image can be used later for evidence purposes.

The storage methods may include on device storage, or synchronizationwith centralized data repository such as a GIS system, or through othermeans that would allow for the storage and retrieval of spatial data.

FIGS. 6 to 8 illustrate example screenshots of an example implementationof the system 150; in this example, validating the position of a pipelocated beneath a manhole. FIG. 6 illustrates an example screenshot ofthe display module 174 displaying a 3D model visual representation ofthe pipe 502 with the positioning as dictated by the object definitionreceived by the object module 170. As illustrated, the background of thescreen can be the mediated reality ‘live’ view received from the camerathat is oriented in the direction of the system 150. In this case, theuser would indicate a discrepancy with the placement of the geospatialobject in the physical space because the visual representation is nocollocated with the corresponding manhole cover. In this example, thevisual representation 502 represents a manhole pipe that should bealigned with, and located subsurface to, the manhole cover 504 capturedby the camera. FIG. 7 illustrates an example screenshot of the visualrepresentation 502 in which the user would validate the objectdefinition as comprising the correct position of the visualrepresentation 502 relative to the corresponding geospatial object(manhole cover) 504 captured by the camera. FIG. 8 illustrates anexample screenshot after the position of the pipe 502 has beenvalidated.

In some cases, the validation module 176 can also receive input of froma user validating attributes associated with the geospatial object viathe user interface 156. These attributes can be part of the objectdefinition received by the object module 170. The recordation module 178can store these validated attributes as metadata associated with thecollected object. These attributes can include, for example, elementssuch as colour, material type, shape, installation date, and the like.In some cases, the system can also automatically capture complementaryattributes, for example, date of the validation, person who performedthe validation, equipment used, and the like. In some cases, thesecomplementary attributes can be validated using MV and/or AI.

In some cases, the system 150 can be initiated from an external systemthat provides instructions to begin. Examples of such external systemscan include ticket management or spatial tracking systems. In anexample, a technician may be reviewing a work ticket using a third-partyticket management system, and as part of the ticket workflow, theexternal system may launch the system 150 for the technician to completethe assignment. In another example, a technician may be passing throughan area for which they will need to collect spatial information. Upondetecting the technician's location, a process may notify the technicianabout the assignment and automatically launch the system 150.

Advantageously, the present embodiments can reduce costs and speed upgeospatial data collection by providing a quicker and more efficient wayto validate the data collection.

While the forgoing refers to a camera to capture a physical scene and ascreen to display the mixture of physical and visual representations, itis contemplated that any apparatus for blending virtual and real objectscan be used; for example, a holographic system that displays holographicaugmentation or projects holograms.

Although the foregoing has been described with reference to certainspecific embodiments, various modifications thereto will be apparent tothose skilled in the art without departing from the spirit and scope ofthe invention as outlined in the appended claims. The entire disclosuresof all references recited above are incorporated herein by reference.

1. A computer-implemented method of validating geospatial object datacollection with mediated reality, the method comprising: receiving anobject definition associated with a geospatial object, the objectdefinition comprising a type of object and a position; displaying avisual representation of the geospatial object to a user relative to acorresponding geospatial object located in a physical scene; andreceiving input validating a placement of the visual representationrelative to the corresponding geospatial object located in the physicalscene.
 2. The method of claim 1, wherein the position of the geospatialobject in the object definition comprises latitude, longitude, andelevation.
 3. The method of claim 2, wherein the position of thegeospatial object in the object definition comprises a point associatedwith the geospatial object.
 4. The method of claim 3, wherein theposition of the geospatial object is determined using at least one ofglobal navigation satellite systems (GNSS) and real-time kinematic (RTK)positioning.
 5. The method of claim 1, wherein receiving inputvalidating the placement of the visual representation comprisesreceiving a confirmatory input from a user.
 6. The method of claim 1,wherein receiving input validating the placement of the visualrepresentation comprises receiving a confirmatory output from machinevision and artificial intelligence techniques.
 7. The method of claim 6,further comprising recording, in the object definition, the validatedposition of the geospatial object.
 8. The method of claim 1, wherein theobject definition further comprises one or more attributes associatedwith the geospatial object, and wherein the method further comprisesreceiving input validating one or more of the attributes associated withthe geospatial object.
 9. The method of claim 8, further comprisingrecording, in the object definition, the one or more validatedattributes associated with the geospatial object.
 10. The method ofclaim 1, further comprising associating an image of the physical scenewith the validated position.
 11. A system of validating geospatialobject data collection with mediated reality, the system comprising oneor more processors and data storage memory in communication with the oneor more processors, the one or more processors configured to execute: anobject module to receive an object definition associated with ageospatial object, the object definition comprising a type of object anda position; a display module to display a visual representation of thegeospatial object to a user relative to a corresponding geospatialobject located in a physical scene; a validation module to receive inputvalidating a placement of the visual representation relative to thecorresponding geospatial object located in the physical scene.
 12. Thesystem of claim 11, wherein the position of the geospatial object in theobject definition comprises latitude, longitude, and elevation.
 13. Thesystem of claim 12, wherein the position of the geospatial object in theobject definition comprises a point associated with the geospatialobject.
 14. The system of claim 13, wherein the position of thegeospatial object is determined using at least one of global navigationsatellite systems (GNSS) and real-time kinematic (RTK) positioning. 15.The system of claim 11, wherein receiving the input validating theplacement of the visual representation comprises receiving aconfirmatory input from a user via an input device.
 16. The system ofclaim 11, wherein receiving input validating the placement of the visualrepresentation comprises receiving a confirmatory output from machinevision and artificial intelligence techniques.
 17. The system of claim16, further comprising a recordation module to record, in the objectdefinition, the validated position of the geospatial object.
 18. Thesystem of claim 11, wherein the object definition further comprises oneor more attributes associated with the geospatial object, and whereinthe validation module further receives input validating one or more ofthe attributes associated with the geospatial object.
 19. The system ofclaim 18, further comprising a recordation module to record, in theobject definition, the one or more validated attributes associated withthe geospatial object.
 20. The system of claim 11, further comprising arecordation module to record an image of the physical scene associatedwith the validated position.